Chunker and Shallow Parser for Free Word Order Languages: An Approach based on Valency Theory and Feature Structures

Free word order languages have relatively unrestricted local word group or phrase structures that make the problem of chunking quite challenging. On the other hand, a robust chunker can drastically reduce the complexity of a parser that follows. We present here a computational framework for chunking of free word order languages based on a generalization of the valency theory. Every word has certain valency constraints that allow it to form local word groups with the adjacent words. The groups so formed have their own valency constraints and thus can recursively participate in the grouping process. This is implemented through feature structure unification. Sentence level constraints on the groups help the chunker reject invalid groupings and detect errors in POS tags provided by a POS-tagger that precedes the chunker. Based on this approach, a Bengali chunker has been implemented with a reasonably good accuracy. The paper also describes how this method can be generalized to develop a complete parser for free word order languages by incorporating semantic information as well as probabilistic models.